We investigate the problem of learning category-specific 3D shape reconstruction from a variable number of RGB views of previously unobserved object instances. Most approaches for multiview shape reconstruction operate on sparse shape representations, or assume a fixed number of views. We present a method that can estimate dense 3D shape, and aggregate shape across multiple and varying number of input views. Given a single input view of an object instance, we propose a representation that encodes the dense shape of the visible object surface as well as the surface behind line of sight occluded by the visible surface. When multiple input views are available, the shape representation is designed to be aggregated into a single 3D shape using an inexpensive union operation. We train a 2D CNN to learn to predict this representation from a variable number of views (1 or more). We further aggregate multiview information by using permutation equivariant layers that promote order-agnostic view information exchange at the feature level. Experiments show that our approach is able to produce dense 3D reconstructions of objects that improve in quality as more views are added.

We thank the reviewers for their valuable comments, and are happy to see feedback such as "concepts presented The reviewers agree that NOX maps are "definitely novel We answer questions, address factual errors, and present more details to improve our manuscript. The differences in chairs between Table 4 and 5 are due to different experimental setting. "Fixed Multi" models were trained with 2, 3, or 5 views respectively. We will add this detail in the paper. "Dated" methods are still valid prior art to compare against especially when We choose to only use the first and last intersections due to computational efficiency.

We investigate the problem of learning category-specific 3D shape reconstruction from a variable number of RGB views of previously unobserved object instances. Most approaches for multiview shape reconstruction operate on sparse shape representations, or assume a fixed number of views. We present a method that can estimate dense 3D shape, and aggregate shape across multiple and varying number of input views. Given a single input view of an object instance, we propose a representation that encodes the dense shape of the visible object surface as well as the surface behind line of sight occluded by the visible surface. When multiple input views are available, the shape representation is designed to be aggregated into a single 3D shape using an inexpensive union operation.

Summary: The authors address the problem of learning category-specific shape reconstruction using the proposed NOX representation. The NOX representation builds on the NOCS idea of normalized object coordinate systems which represents all instances in an object category within a unit cube. Predicting a perspective projection of the NOCS representation in the camera view (called the NOCS map) is thus equivalent to predicting the object shape coordinates in the unit cube (or NOCS). The authors extend this to not just predict NOCS coordinates of the visible surface in a camera view (first intersection of ray from pixel to object) but also coordinates of the the "last* intersection of the ray. This pair of first and last intersection maps termed NOX thus provide a reasonably complete picture of object shape (for mostly convex objects).

We investigate the problem of learning category-specific 3D shape reconstruction from a variable number of RGB views of previously unobserved object instances. Most approaches for multiview shape reconstruction operate on sparse shape representations, or assume a fixed number of views. We present a method that can estimate dense 3D shape, and aggregate shape across multiple and varying number of input views. Given a single input view of an object instance, we propose a representation that encodes the dense shape of the visible object surface as well as the surface behind line of sight occluded by the visible surface. When multiple input views are available, the shape representation is designed to be aggregated into a single 3D shape using an inexpensive union operation.

We investigate the problem of learning category-specific 3D shape reconstruction from a variable number of RGB views of previously unobserved object instances. Most approaches for multiview shape reconstruction operate on sparse shape representations, or assume a fixed number of views. We present a method that can estimate dense 3D shape, and aggregate shape across multiple and varying number of input views. Given a single input view of an object instance, we propose a representation that encodes the dense shape of the visible object surface as well as the surface behind line of sight occluded by the visible surface. When multiple input views are available, the shape representation is designed to be aggregated into a single 3D shape using an inexpensive union operation.